Augmented reality visual rendering systems comprising at least one partially transparent display device, such as “Google glasses”, are known, these being described in a patent application US 2013/0044042. Augmented reality visual rendering systems such as augmented reality windshields are also known.
Such systems often comprise partially transparent display devices often loosely referred to as semi-transparent display devices.
Partially transparent display systems, such as semi-transparent glasses, semi-transparent screens, systems for projection onto semi-transparent surfaces, etc., make it possible to augment the user's view by inserting (augmented reality) or deleting (diminished reality) visual elements in the user's field of vision. The benefit of these semi-transparent display devices resides notably in the fact that the user continues to perceive the real environment with no latency time and according to the same viewpoint as the latter would have without the display device.
Nonetheless, numerous applications require that the information displayed on the partially transparent device be aligned with real elements of the scene according to the user's viewpoint. For example, the display device can be used to highlight an element of the real scene. The activated display zone must then be projected onto the retina of the user's eye at the same place that the image of the real object is projected. It is then necessary to calibrate the “eye-display device” system so as to afford a model characterizing the projection of the image displayed on the display device on the retina of the eye.
Given that the position of the display device with respect to the eye can vary from one user to another (for example due to the difference in inter-pupillary distance between different users), or indeed from one use to another for one and the same user (slight variation in the position of the glasses on the head), calibration carried out in the factory does not make it possible to obtain optimal alignment of the image displayed on the glasses with the user's natural vision. It is therefore beneficial to enable the user to be able to calibrate or refine a previous calibration, so as to obtain a calibration suitable for the current conditions of use of the device. The main technical problem then consists in providing an ergonomic solution enabling the user to perform a calibration at their workplace or to refine the factory calibration.
There is known, from the document “Spatial calibration of an optical see-through head mounted display” by Stuart J. Gilson, Andrew W. Fitzgibbon and Andrew Glennerster, a scheme for calibrating semi-transparent glasses in which the head of the user is replaced by that of a mannequin comprising a camera in the guise of an eye. The image provided by the camera is used to calibrate the semi-transparent glasses. Accordingly, a calibration testcard is used. The testcard is placed in front of the mannequin head and then detected in the image captured by the camera. Though the use of a camera instead of the eye makes it possible to use computer-vision algorithms to detect a calibration testcard and calibrate the system, this approach nonetheless exhibits significant limitations. In particular, this scheme assumes that the user's eye is sited at the same place as the camera whilst the glasses can be worn by users, the morphology of whose head may vary. Moreover, the calibration process requires a mannequin head equipped with a camera. Though this constraint is compatible with in-factory calibration, it is not compatible with calibration carried out away from the factory.
It is also known to require user interactions in order to perform a calibration. Thus, the system successively displays points in a pair of glasses, and the user must align each of these 2D points (of a two-dimensional space of the display device) with a 3D point (in the user's space) whose coordinates are known in the reference frame of the user's head.
The document “Calibrating an optical see-through rig with two non-overlapping cameras: The virtual camera framework” by J. Braux-Zin, A. Bartoli, and R. Dupont proposes to calibrate a semi-transparent screen by using a calibration testcard exhibiting 3D points of known coordinates. The user is then invited to move a pointer on the screen in such a way as to designate the position on the screen where each 3D point of the testcard is perceived by the user. This procedure exhibits several drawbacks. Firstly, in order for the calibration to be accurate over the whole of the screen, the testcard used must cover the whole of the screen or be moved in tandem with the calibration process. Such a procedure is therefore conceivable within the framework of an in-factory calibration but is somewhat ill-adapted for a calibration procedure carried out at the spot where the device is utilized. Moreover, this requires that a device for pointing at the screen of mouse/touchpad type be available, this not always being the case, in particular in the case of semi-transparent glasses.
Document US 2002/0105484 (Navab and Tuceryan) proposes to use a 3D point whose position is fixed in the environment, and it is then up to the user to move their head so as to align the 2D point with the image of the 3D point perceived by the user's eye. It also proposes to use a mobile stylus whose tip is located in 3D that the user is invited to move in such a way as to align the tip of the latter with the 2D point displayed in the glasses. In both cases, when the user estimates that the 3D point is aligned with the 2D point displayed in the glasses, the latter indicates this to the system and the association between the 3D point and the 2D point is stored. When a sufficient number of such associations is available, the calibration is carried out with the aid of a conventional algorithm for camera calibration. Though this type of approach makes it possible to calibrate the device for the user's eye, the procedure used exhibits in particular the drawback of being rather unergonomic. Indeed, the scheme used being based on a successive capture of positions of 3D points, this requires the user to remain motionless when he considers that the 3D point is correctly aligned with the 2D point displayed in the glasses, doing so for the time it takes to validate the capture. This step, requiring a motionless position of the user, is carried out a large number of times (once for each association between a 3D point and 2D point), this generally involving significant muscular fatigue for the user, and rendering the calibration process lengthy to carry out. This lack of ergonomics may then lead to rejection of the device by the end user.
Stated otherwise, the main limitation of this solution is related to the fact that the user must move their hand and their head so as to align the tip of the stylus with the 2D point displayed, and then remain still for the time it takes to validate the alignment. During this time of motionlessness, the user moves neither their hand nor their head. Accurate calibration makes it necessary to repeat this step for tens of 2D points, thereby rapidly giving rise to muscular fatigue and making the calibration process lengthy.